Multimodal-based and Aesthetic-guided Narrative Video Summarization

Narrative videos usually illustrate the main content through multiple narrative information such as audios, video frames and subtitles. Existing summarization approaches rarely consider dimensional inputs, or ignore impact of shots artistic assembly when directly applied to videos. This paper introduces a multimodal-based aesthetic-guided method. Our method leverages multimodal including visual content, subtitles audio our specified key selection, subtitle summarization, highlight extraction components. Furthermore, under guidance cinematographic aesthetic, we design novel module ensure shot completeness then assemble selected into desired summary. Besides, also provides flexible specification for achieve which it automatically selects semantically related according user-designed text. By conducting large number quantitative experimental evaluations user studies, demonstrate that effectively preserves important original video, is capable rapidly producing high-quality summaries.